Information processing apparatus, object recognition apparatus, device control system, movable body, image processing method, and computer-readable recording medium

ABSTRACT

According to an embodiment, an information processing apparatus includes a calculating unit and a discarding unit. The calculating unit is configured to calculate a distance between two objects, detected based on distance information on the objects that are overlapped in detection areas of the objects, in a depth direction in the detection areas. The discarding unit is configured to determine whether each of the two objects in the detection areas is to be discarded by using a method that corresponds to the distance calculated by the calculating unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT international application Ser.No. PCT/JP2016/086640 filed on Dec. 8, 2016 which designates the UnitedStates, incorporated herein by reference, and which claims the benefitof priority from Japanese Patent Applications No. 2016-051447, filed onMar. 15, 2016, incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

Embodiments relate to an information processing apparatus, an objectrecognition apparatus, a device control system, a movable body, an imageprocessing method, and a computer-readable recording medium.

2. Description of the Related Art

Conventionally, body structures of automobiles, and the like, have beendeveloped in terms of safety of automobiles as to how pedestrians andoccupants in an automobile are protected when the automobile crashesinto pedestrians. Furthermore, in recent years, technologies ofdetecting persons and automobiles at high speed have been developed dueto improvements in information processing technologies and imageprocessing technologies. By using these technologies, some automobileshave been developed to prevent crashes before happens by automaticallyapplying a brake before an automobile crashes into an object. Forautomatic control of automobiles, the distance to an object such asperson or different automobile needs to be measured with accuracy and,for this purpose, distance measurement using millimeter-wave radar andlaser radar, distance measurement using a stereo camera, and the like,have been put into practical use.

When a stereo camera is used as a technology for recognizing objects,disparity of each object appearing in two luminance images captured onthe right and left is derived to generate a disparity image and pixelshaving a similar disparity value is grouped together to recognize theobject. Here, by extracting a disparity cluster in a disparity image,the height, horizontal width, and depth of an object and the position ofan object in three dimensions may be detected.

As the technology for recognizing objects described above, there is adisclosed technology in which a pedestrian recognition area where thepresence of a pedestrian is recognized in image data is identified and apedestrian score indicating the degree of certainty of a pedestrian iscalculated (see Japanese Laid-open Patent Publication No. 2014-146267).

Typically, when objects are overlapped in a captured image, a process isconducted to exclude (discard) an object in the back from the controltarget (tracking target); however, it is preferable that, for example,pedestrians who run out from the back side of a different vehicle in thefront are not discarded but included as the control target.Unfortunately, the technology disclosed in Japanese Laid-open PatentPublication No. 2014-146267 has a problem in that for example when apedestrian suddenly runs out from the back of a different vehicle, orthe like, it is difficult to ensure that the pedestrian is detectedwithout being discarded and is included as the control target.

In consideration of the foregoing, there is a need to provide aninformation processing apparatus, an object recognition apparatus, adevice control system, a movable body, an image processing method, and acomputer-readable recording medium having a program that performs adiscard process properly.

SUMMARY OF THE INVENTION

According to an embodiment, an information processing apparatus includesa calculating unit and a discarding unit. The calculating unit isconfigured to calculate a distance between two objects, detected basedon distance information on the objects that are overlapped in detectionareas of the objects, in a depth direction in the detection areas. Thediscarding unit is configured to determine whether each of the twoobjects in the detection areas is to be discarded by using a method thatcorresponds to the distance calculated by the calculating unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are diagrams that illustrate an example where a devicecontrol system according to an embodiment is installed in a vehicle;

FIG. 2 is a diagram that illustrates an example of the externalappearance of an object recognition apparatus according to theembodiment;

FIG. 3 is a diagram that illustrates an example of the hardwareconfiguration of the object recognition apparatus according to theembodiment;

FIG. 4 is a diagram that illustrates an example of the configuration offunctional blocks of the object recognition apparatus according to theembodiment;

FIG. 5 is a diagram that illustrates an example of the configuration offunctional blocks in a disparity-value calculation processing unit ofthe object recognition apparatus according to the embodiment;

FIG. 6 is a diagram that explains the principle for deriving thedistance from an imaging unit to an object;

FIG. 7 is a diagram that explains the case of obtaining a correspondingpixel that is in a comparison image and that corresponds to thereference pixel in the reference image;

FIG. 8 is a diagram that illustrates an example of the graph of resultsof block matching processing;

FIG. 9 is a diagram that illustrates an example of the configuration offunctional blocks of the recognition processing unit in the objectrecognition apparatus according to the embodiment;

FIG. 10 is a diagram that illustrates an example of the V map generatedfrom a disparity image;

FIG. 11 is a diagram that illustrates an example of the U map generatedfrom a disparity image;

FIG. 12 is a diagram that illustrates an example of the real U mapgenerated from a U map;

FIG. 13 is a diagram that illustrates a process to extract an isolatedarea from a real U map;

FIG. 14 is a diagram that illustrates a process to generate a detectionframe;

FIG. 15 is a diagram that illustrates a case where the distance betweenframes is short;

FIG. 16 is a diagram that illustrates a case where the distance betweenframes is long;

FIG. 17 is a flowchart that illustrates an example of operation duringblock matching processing by a disparity-value deriving unit accordingto the embodiment;

FIG. 18 is a flowchart that illustrates an example of operation duringthe object recognition process by a recognition processing unitaccording to the embodiment;

FIG. 19 is a flowchart that illustrates an example of operation duringthe overlap process by the recognition processing unit according to theembodiment;

FIG. 20 is a diagram that illustrates an overlap size when the distancebetween frames is a short distance;

FIG. 21 is a diagram that illustrates operation to discard a detectionobject when the distance between frames is a short distance;

FIG. 22 is a diagram that illustrates an overlap size when the distancebetween frames is a long distance;

FIG. 23 is a diagram that illustrates a case where there is no overlapsize when the distance between frames is a long distance; and

FIG. 24 is a diagram that illustrates a case where a detection object isnot discarded when the distance between frames is a long distance.

The accompanying drawings are intended to depict exemplary embodimentsof the present invention and should not be interpreted to limit thescope thereof. Identical or similar reference numerals designateidentical or similar components throughout the various drawings.

DESCRIPTION OF THE EMBODIMENTS

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentinvention.

As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise.

In describing preferred embodiments illustrated in the drawings,specific terminology may be employed for the sake of clarity. However,the disclosure of this patent specification is not intended to belimited to the specific terminology so selected, and it is to beunderstood that each specific element includes all technical equivalentsthat have the same function, operate in a similar manner, and achieve asimilar result.

With reference to FIGS. 1A to 24, a detailed explanation is given belowof an embodiment of an image processing apparatus, an object recognitionapparatus, a device control system, an image processing method, and aprogram according to the present invention. The present invention is notlimited to the embodiment below, and components in the embodiment belowinclude the ones that may be easily developed by a person skilled in theart, substantially the same ones, and the ones in what is called a rangeof equivalents. Furthermore, the components may be variously omitted,replaced, modified, or combined without departing from the scope of theembodiment below.

[Schematic Configuration of Vehicle Including Object RecognitionApparatus]

FIGS. 1A and 1B are diagrams that illustrate an example where a devicecontrol system according to the embodiment is installed in a vehicle.With reference to FIGS. 1A and 1B, an explanation is given of a casewhere for example a device control system 60 according to the presentembodiment is installed in a vehicle 70.

FIG. 1A is a side view of the vehicle 70 with the device control system60 installed therein, and FIG. 1B is a front view of the vehicle 70.

As illustrated in FIGS. 1A and 1B, the vehicle 70, which is anautomobile, has the device control system 60 installed therein. Thedevice control system 60 includes an object recognition apparatus 1, avehicle control device 6 (control device), a steering wheel 7, and abrake pedal 8, provided in the vehicle interior that is an accommodationspace in the vehicle 70.

The object recognition apparatus 1 has an imaging function to captureimages in a traveling direction of the vehicle 70, and for example it isinstalled near the rearview mirror inside the front window of thevehicle 70. The object recognition apparatus 1 includes: a main bodyunit 2; and an imaging unit 10 a and an imaging unit 10 b that are fixedto the main body unit 2, and details of its configuration and operationare described later. The imaging units 10 a, 10 b are fixed to the mainbody unit 2 so as to capture an object in the traveling direction of thevehicle 70.

The vehicle control device 6 is an ECU (electronic control unit) thatperforms various types of vehicle control on the basis of recognitioninformation received from the object recognition apparatus 1. On thebasis of recognition information received from the object recognitionapparatus 1, the vehicle control device 6 performs, as an example of thevehicle control, steering control to avoid obstacles by controlling asteering system (control target) including the steering wheel 7, brakecontrol to stop or reduce the speed of the vehicle 70 by controlling thebrake pedal 8 (control target), or the like.

The device control system 60 including the object recognition apparatus1 and the vehicle control device 6 described above performs vehiclecontrol such as steering control or brake control to improve drivingsafety of the vehicle 70.

Furthermore, as described above, the object recognition apparatus 1captures images in front of the vehicle 70; however, this is not alimitation. That is, the object recognition apparatus 1 may be installedto capture images on the back or side of the vehicle 70. In this case,the object recognition apparatus 1 is capable of detecting the positionof the following vehicle and person on the back of the vehicle 70 or adifferent vehicle and person on the side thereof. Furthermore, thevehicle control device 6 is capable of detecting dangers when thevehicle 70 changes a lane, merges into a lane, or the like, to performthe above-described vehicle control. Furthermore, when the vehiclecontrol device 6 determines that there is the danger of collision whenthe vehicle 70 is backing to be parked, or the like, on the basis ofrecognition information on an obstacle on the back of the vehicle 70,output from the object recognition apparatus 1, it is capable ofperforming the above-described vehicle control.

[Configuration of the Object Recognition Apparatus]

FIG. 2 is a diagram that illustrates an example of the externalappearance of the object recognition apparatus according to theembodiment. As illustrated in FIG. 2, the object recognition apparatus 1includes the main body unit 2; and the imaging unit 10 a and the imagingunit 10 b that are fixed to the main body unit 2, as described above.The imaging units 10 a and 10 b are made up of a pair of cylindricalcameras that are parallel to and are located at equivalent positionsrelative to the main body unit 2. Furthermore, for the convenience ofexplanation, the imaging unit 10 a illustrated in FIG. 2 is sometimesreferred to as the right camera and the imaging unit 10 b as the leftcamera.

(Hardware Configuration of the Object Recognition Apparatus)

FIG. 3 is a diagram that illustrates an example of the hardwareconfiguration of the object recognition apparatus according to theembodiment. With reference to FIG. 3, the hardware configuration of theobject recognition apparatus 1 is explained.

As illustrated in FIG. 3, the object recognition apparatus 1 includes adisparity-value deriving unit 3 and a recognition processing unit 5inside the main body unit 2.

The disparity-value deriving unit 3 is a device that derives a disparityvalue dp indicating disparity with respect to an object from imagesobtained after the object is captured and outputs a disparity image (anexample of distance information) indicating the disparity value dp ofeach pixel. The recognition processing unit 5 is a device that performsan object recognition process, or the like, on an object such as personor vehicle appearing in a captured image on the basis of a disparityimage output from the disparity-value deriving unit 3 and outputsrecognition information that is information indicating a result of theobject recognition process to the vehicle control device 6.

As illustrated in FIG. 3, the disparity-value deriving unit 3 includesthe imaging unit 10 a, the imaging unit 10 b, a signal converting unit20 a, a signal converting unit 20 b, and an image processing unit 30.

The imaging unit 10 a is a processing unit that captures an object inthe front and generates analog image signals. The imaging unit 10 aincludes an imaging lens 11 a, an aperture 12 a, and an image sensor 13a.

The imaging lens 11 a is an optical element that refracts incident lightto form an image of the object on the image sensor 13 a. The aperture 12a is a member that blocks part of light that has passed through theimaging lens 11 a to adjust the amount of light input to the imagesensor 13 a. The image sensor 13 a is a semiconductor device thatconverts light that has entered the imaging lens 11 a and passed throughthe aperture 12 a into electric analog image signals. The image sensor13 a is implemented by using solid state image sensors such as CCD(charge coupled devices) or CMOS (complementary metal oxidesemiconductor).

The imaging unit 10 b is a processing unit that captures the object inthe front and generates analog image signals. The imaging unit 10 bincludes an imaging lens 11 b, an aperture 12 b, and an image sensor 13b. Here, the functions of the imaging lens 11 b, the aperture 12 b, andthe image sensor 13 b are the same as those of the imaging lens 11 a,the aperture 12 a, and the image sensor 13 a described above.Furthermore, the imaging lens 11 a and the imaging lens 11 b areinstalled such that their principal surfaces are on the same plane sothat the right and the left cameras capture images under the samecondition.

The signal converting unit 20 a is a processing unit that convertsanalog image signals generated by the imaging unit 10 a intodigital-format image data. The signal converting unit 20 a includes aCDS (correlated double sampling) 21 a, an AGC (auto gain control) 22 a,an ADC (analog digital converter) 23 a, and a frame memory 24 a.

The CDS 21 a removes noise from analog image signals generated by theimage sensor 13 a by using correlated double sampling, a differentialfilter in a traverse direction, a smoothing filter in a longitudinaldirection, or the like. The AGC 22 a performs gain control to controlthe intensity of analog image signals from which noise has been removedby the CDS 21 a. The ADC 23 a converts analog image signals whose gainhas been controlled by the AGC 22 a into digital-format image data. Theframe memory 24 a stores image data converted by the ADC 23 a.

The signal converting unit 20 b is a processing unit that convertsanalog image signals generated by the imaging unit 10 b intodigital-format image data. The signal converting unit 20 b includes aCDS 21 b, an AGC 22 b, an ADC 23 b, and a frame memory 24 b. Here, thefunctions of the CDS 21 b, the AGC 22 b, the ADC 23 b, and the framememory 24 b are the same as those of the CDS 21 a, the AGC 22 a, the ADC23 a, and the frame memory 24 a described above.

The image processing unit 30 is a device that performs image processingon image data converted by the signal converting unit 20 a and thesignal converting unit 20 b. The image processing unit 30 includes anFPGA (field programmable gate array) 31, a CPU (central processing unit)32, a ROM (read only memory) 33, a RAM (random access memory) 34, an I/F(interface) 35, and a bus line 39.

The FPGA 31 is an integrated circuit, and here it performs a process toderive the disparity value dp in an image based on image data. The CPU32 controls each function of the disparity-value deriving unit 3. TheROM 33 stores programs for image processing executed by the CPU 32 tocontrol each function of the disparity-value deriving unit 3. The RAM 34is used as a work area of the CPU 32. The I/F 35 is an interface forcommunicating with an I/F 55 in the recognition processing unit 5 via acommunication line 4. As illustrated in FIG. 3, the bus line 39 is anaddress bus and a data bus, or the like, for connecting the FPGA 31, theCPU 32, the ROM 33, the RAM 34, and the I/F 35 such that they cancommunicate with one other.

Here, the image processing unit 30 includes the FPGA 31 as an integratedcircuit for deriving the disparity value dp; however, this is not alimitation, and it may be an integrated circuit such as ASIC(application specific integrated circuit).

As illustrated in FIG. 3, the recognition processing unit 5 includes anFPGA 51, a CPU 52, a ROM 53, a RAM 54, the I/F 55, a CAN (controllerarea network) I/F 58, and a bus line 59.

The FPGA 51 is an integrated circuit, and here it performs an objectrecognition process on an object on the basis of disparity images, orthe like, received from the image processing unit 30. The CPU 52controls each function of the recognition processing unit 5. The ROM 53stores programs for an object recognition process with which the CPU 52performs an object recognition process of the recognition processingunit 5. The RAM 54 is used as a work area of the CPU 52. The I/F 55 isan interface for data communication with the I/F 35 of the imageprocessing unit 30 via the communication line 4. The CAN I/F 58 is aninterface for communicating with an external controller (e.g., thevehicle control device 6 illustrated in FIG. 3), and, for example, a busline 59 connected to the CAN of a vehicle, or the like, is an addressbus and a data bus, or the like, connecting the FPGA 51, the CPU 52, theROM 53, the RAM 54, the I/F 55, and the CAN I/F 58 such that they cancommunicate with one another, as illustrated in FIG. 3.

With this configuration, after a disparity image is sent to therecognition processing unit 5 from the I/F 35 of the image processingunit 30 via the communication line 4, the FPGA 51 performs an objectrecognition process, or the like, on an object such as person or vehicleappearing in a captured image on the basis of the disparity image inaccordance with a command from the CPU 52 of the recognition processingunit 5.

Furthermore, each of the above-described programs may be distributed byrecorded in a recording medium readable by computers in the form of filethat is installable and executable. The recording medium may be a CD-ROM(compact disc read only memory), SD (secure digital) memory card, or thelike.

Furthermore, as illustrated in FIG. 3, the image processing unit 30 ofthe disparity-value deriving unit 3 and the recognition processing unit5 are separate devices; however, this is not a limitation, and, forexample, the image processing unit 30 and the recognition processingunit 5 may be the same device to generate disparity images and performan object recognition process.

(Configuration and Operation of Functional Blocks of the ObjectRecognition Apparatus)

FIG. 4 is a diagram that illustrates an example of the configuration offunctional blocks of the object recognition apparatus according to theembodiment. First, with reference to FIG. 4, an explanation is given ofthe configuration and operation of the functional blocks in the relevantpart of the object recognition apparatus 1.

Although described above with reference to FIG. 3, the objectrecognition apparatus 1 includes the disparity-value deriving unit 3 andthe recognition processing unit 5 as illustrated in FIG. 4.Specifically, the disparity-value deriving unit 3 includes an imageacquiring unit 100 a (first imaging unit), an image acquiring unit 100 b(second imaging unit), converting units 200 a, 200 b, and adisparity-value calculation processing unit 300 (generating unit).

The image acquiring unit 100 a is a functional unit that captures theimage of an object in the front by using the right camera, generatesanalog image signals, and obtains a luminance image that is an imagebased on the image signals. The image acquiring unit 100 a isimplemented by using the imaging unit 10 a illustrated in FIG. 3.

The image acquiring unit 100 b is a functional unit that captures theimage of an object in the front by using the left camera, generatesanalog image signals, and obtains a luminance image that is an imagebased on the image signals. The image acquiring unit 100 b isimplemented by using the imaging unit 10 b illustrated in FIG. 3.

The converting unit 200 a is a functional unit that removes noise fromimage data on the luminance image obtained by the image acquiring unit100 a, converts it into digital-format image data, and outputs it. Theconverting unit 200 a is implemented by using the signal converting unit20 a illustrated in FIG. 3.

The converting unit 200 b is a functional unit that removes noise fromimage data on the luminance image obtained by the image acquiring unit100 b, converts it into digital-format image data, and outputs it. Theconverting unit 200 b is implemented by using the signal converting unit20 b illustrated in FIG. 3.

Here, with regard to pieces of image data (hereafter, simply referred toas luminance images) on two luminance images output from the convertingunits 200 a, 200 b, the luminance image captured by the image acquiringunit 100 a, which is the right camera (the imaging unit 10 a), is theimage data on a reference image Ia (hereafter, simply referred to as thereference image Ia) (first captured image), and the luminance imagecaptured by the image acquiring unit 100 b, which is the left camera(the imaging unit 10 b), is the image data on a comparison image Ib(hereafter, simply referred to as the comparison image Ib) (secondcaptured image). That is, the converting units 200 a, 200 b output thereference image Ia and the comparison image Ib, respectively, on thebasis of two luminance images output from the image acquiring units 100a, 100 b.

The disparity-value calculation processing unit 300 is a functional unitthat derives the disparity value dp with respect to each pixel of thereference image Ia on the basis of the reference image Ia and thecomparison image Ib received from the converting units 200 a, 200 b,respectively, and generates a disparity image in which the disparityvalue dp is applied to each pixel of the reference image Ia. Thedisparity-value calculation processing unit 300 outputs the generateddisparity image to the recognition processing unit 5.

The recognition processing unit 5 is a functional unit that recognizes(detects) an object on the basis of the reference image Ia and thedisparity image received from the disparity-value deriving unit 3 andperforms a tracking process on the recognized object.

Configuration and Operation of Functional Blocks of the Disparity-ValueCalculation Processing Unit

FIG. 5 is a diagram that illustrates an example of the configuration offunctional blocks in the disparity-value calculation processing unit ofthe object recognition apparatus according to the embodiment. FIG. 6 isa diagram that explains the principle for deriving the distance from theimaging unit to an object. FIG. 7 is a diagram that explains the case ofobtaining a corresponding pixel that is in a comparison image and thatcorresponds to the reference pixel in the reference image. FIG. 8 is adiagram that illustrates an example of the graph of results of blockmatching processing.

First, with reference to FIGS. 6 to 8, a distance measuring method usingblock matching processing is schematically explained.

Principle of Distance Measurement

With reference to FIG. 6, an explanation is given of the principle ofderiving the disparity with respect to an object from the stereo cameradue to stereo matching processing and measuring the distance from thestereo camera to the object by using the disparity value representingthe disparity.

The imaging system illustrated in FIG. 6 includes the imaging unit 10 aand the imaging unit 10 b that are located parallel at equivalentpositions. The imaging units 10 a, 10 b include the imaging lens 11 a,11 b, respectively, which refract incident light to form an image of theobject on an image sensor that is a solid state image sensor. Imagescaptured by the imaging unit 10 a and the imaging unit 10 b are thereference image Ia and the comparison image Ib, respectively. In FIG. 6,on each of the reference image Ia and the comparison image Ib, a point Son an object E in the three-dimensional space is mapped onto a positionon a straight line parallel to the straight line connecting the imaginglens 11 a and the imaging lens 11 b. Here, the point S mapped onto eachimage is a point Sa(x,y) on the reference image Ia and is a pointSb(X,y) on the comparison image Ib. Here, the disparity value dp isrepresented as in Equation (1) below by using the point Sa(x,y) oncoordinates of the reference image Ia and the point Sb(X,y) oncoordinates of the comparison image Ib.dp=X−x  (1)

Furthermore, in FIG. 6, the disparity value dp may be represented asdp=Δa+Δb, where Δa is the distance between the point Sa(x,y) on thereference image Ia and the intersection point of the perpendicularextending from the imaging lens 11 a with the imaging surface and Δb isthe distance between the point Sb(X,y) on the comparison image Ib andthe intersection point of the perpendicular extending from the imaginglens 11 b with the imaging surface.

Then, by using the disparity value dp, a distance Z between the imagingunits 10 a, 10 b and the object E is derived. Here, the distance Z isthe distance from the straight line connecting the focus position of theimaging lens 11 a and the focus position of the imaging lens 11 b to thepoint S on the object E. As illustrated in FIG. 6, the distance Z may becalculated with Equation (2) below by using a focal length f of theimaging lens 11 a and the imaging lens 11 b, a base length B that is thedistance between the imaging lens 11 a and the imaging lens 11 b, andthe disparity value dp.Z=(B×f)/dp  (2)

According to Equation (2), it is understood that the distance Z isshorter as the disparity value dp is larger and the distance Z is longeras the disparity value dp is smaller.

Block Matching Processing

Next, with reference to FIGS. 7 and 8, an explanation is given of adistance measuring method due to block matching processing.

With reference to FIGS. 7 and 8, a method of calculating a cost valueC(p,d) is explained. In the following explanation, C(p,d) representsC(x,y,d).

As for FIG. 7, a section (a) is a conceptual diagram that illustrates areference pixel p and a reference area pb in the reference image Ia, anda section (b) is a conceptual diagram of calculating the cost value Cwhile sequentially shifting (displacing) candidates for thecorresponding pixel that is in the comparison image Ib and thatcorresponds to the reference pixel p illustrated in the section (a) ofFIG. 7. Here, the corresponding pixel indicates the pixel that is in thecomparison image Ib and that is nearest to the reference pixel p in thereference image Ia. Furthermore, the cost value C is an evaluation value(degree of matching) representing the degree of similarity or the degreeof dissimilarity of each pixel in the comparison image Ib with respectto the reference pixel p in the reference image Ia. In explanation, thecost value C described below is an evaluation value representing thedegree of dissimilarity indicating that as the value is smaller, thepixel in the comparison image Ib is similar to the reference pixel p.

As illustrated in the section (a) of FIG. 7, on the basis of theluminance value (pixel value) of the reference pixel p(x,y) in thereference image Ia and each of the candidate pixels q(x+d,y) that arecandidates for the corresponding pixel on an epipolar line EL in thecomparison image Ib with respect to the reference pixel p(x,y), the costvalue C(p,d) of the candidate pixel q(x+d,y) that is a candidate for thecorresponding pixel with respect to the reference pixel p(x,y) iscalculated. The shift amount (displacement amount) between the referencepixel p and the candidate pixel q is d, and the shift amount d is ashift on a pixel to pixel basis. Specifically, while the candidate pixelq(x+d,y) is sequentially shifted by one pixel within a predeterminedrange (e.g., 0<d<25), the cost value C(p,d) is calculated, which is thedegree of dissimilarity between the luminance values of the candidatepixel q(x+d,y) and the reference pixel p(x,y). Furthermore, as thestereo matching processing to obtain the corresponding pixel of thereference pixel p, block matching processing is performed according tothe present embodiment. During the block matching processing, the degreeof dissimilarity is obtained between the reference area pb that is apredetermined area with the reference pixel p in the reference image Iaas a center and a candidate area qb (the same size as the reference areapb) with the candidate pixel q in the comparison image Ib as a center.SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), ZSSD(Zero-mean-Sum of Squared Difference), which is obtained by subtractingthe average value of blocks from the value of SSD, or the like, is usedas the cost value C indicating the degree of dissimilarity between thereference area pb and the candidate area qb. These evaluation valuesrepresent the degree of dissimilarity because the value is smaller asthe correlation is higher (the degree of similarity is higher).

Furthermore, as described above, the imaging units 10 a, 10 b arelocated parallel at equivalent positions and therefore the referenceimage Ia and the comparison image Ib also have a relation such that theyare located parallel at equivalent positions. Therefore, thecorresponding pixel that is in the comparison image Ib and thatcorresponds to the reference pixel p in the reference image Ia ispresent on the epipolar line EL that is illustrated as a line in ahorizontal direction as viewed from the sheet surface in FIG. 7 and, toobtain the corresponding pixel in the comparison image Ib, a pixel isretrieved on the epipolar line EL of the comparison image Ib.

The cost value C(p,d) calculated during the above-described blockmatching processing is represented by, for example, the graphillustrated in FIG. 8 in relation to the shift amount d. In the exampleof FIG. 8, as the cost value C is the minimum value when the shiftamount d=7, the disparity value dp=7 is derived.

Specific configuration and operation of functional blocks of thedisparity-value calculation processing unit

With reference to FIG. 5, the specific configuration and operation offunctional blocks of the disparity-value calculation processing unit 300are explained.

As illustrated in FIG. 5, the disparity-value calculation processingunit 300 includes a cost calculating unit 301, a determining unit 302,and a first generating unit 303.

The cost calculating unit 301 is a functional unit that calculates thecost value C(p,d) of each of the candidate pixels q(x+d,y) on the basisof the luminance value of the reference pixel p(x,y) in the referenceimage Ia and the luminance value of each of the candidate pixelsq(x+d,y) that are candidates for the corresponding pixel, identified byshifting the pixel at the corresponding position of the reference pixelp(x,y) by the shift amount d on the epipolar line EL on the comparisonimage Ib based on the reference pixel p(x,y). Specifically, during blockmatching processing, the cost calculating unit 301 calculates, as thecost value C, the degree of dissimilarity between the reference area pbthat is a predetermined area with the reference pixel p in the referenceimage Ia as a center and the candidate area qb (the same size as thereference area pb) with the candidate pixel q in the comparison image Ibas a center.

The determining unit 302 is a functional unit that determines that theshift amount d that corresponds to the minimum value of the cost value Ccalculated by the cost calculating unit 301 is the disparity value dpwith respect to a pixel in the reference image Ia that is targeted forcalculation of the cost value C.

The first generating unit 303 is a functional unit that generates adisparity image that is an image where, on the basis of the disparityvalue dp determined by the determining unit 302, the pixel value of eachpixel of the reference image Ia is replaced with the disparity value dpthat corresponds to the pixel.

Each of the cost calculating unit 301, the determining unit 302, and thefirst generating unit 303 illustrated in FIG. 5 is implemented by usingthe FPGA 31 illustrated in FIG. 3. Furthermore, all or part of the costcalculating unit 301, the determining unit 302, and the first generatingunit 303 may be implemented when the CPU 32 executes programs stored inthe ROM 33 instead of the FPGA 31 that is a hardware circuit.

Here, the functions of the cost calculating unit 301, the determiningunit 302, and the first generating unit 303 in the disparity-valuecalculation processing unit 300 illustrated in FIG. 5 are illustrated asa concept, and this configuration is not a limitation. For example,multiple functional units that are illustrated as separate functionalunits in the disparity-value calculation processing unit 300 illustratedin FIG. 5 may be configured as a single functional unit. Conversely, afunction provided in a single functional unit in the disparity-valuecalculation processing unit 300 illustrated in FIG. 5 may be divided andconfigured as multiple functional units.

Configuration and Operation of Functional Blocks in the RecognitionProcessing Unit

FIG. 9 is a diagram that illustrates an example of the configuration offunctional blocks of the recognition processing unit in the objectrecognition apparatus according to the embodiment. FIG. 10 is a diagramthat illustrates an example of the V map generated from a disparityimage. FIG. 11 is a diagram that illustrates an example of the U mapgenerated from a disparity image. FIG. 12 is a diagram that illustratesan example of the real U map generated from a U map. FIG. 13 is adiagram that illustrates a process to extract an isolated area from areal U map. FIG. 14 is a diagram that illustrates a process to generatea detection frame. FIG. 15 is a diagram that illustrates a case wherethe distance between frames is short. FIG. 16 is a diagram thatillustrates a case where the distance between frames is long. Withreference to FIGS. 9 to 16, the configuration and operation offunctional blocks of the recognition processing unit 5 are explained.

As illustrated in FIG. 9, the recognition processing unit 5 includes asecond generating unit 501, a clustering processing unit 502, and atracking unit 503.

The second generating unit 501 is a functional unit that receives adisparity image from the disparity-value calculation processing unit300, receives the reference image Ia from the disparity-value derivingunit 3, and generates a V-Disparity map, U-Disparity map, and RealU-Disparity map, or the like. Specifically, to detect a road surfacefrom the disparity image input from the disparity-value calculationprocessing unit 300, the second generating unit 501 generates a V map VMthat is the V-Disparity map illustrated in a section (b) of FIG. 10.Here, the V-Disparity map is a two-dimensional histogram indicating thefrequency distribution of the disparity value dp, where the verticalaxis is the y axis of the reference image Ia and the horizontal axis isthe disparity value dp (or distance) of the disparity image. Forexample, a road surface 600, a power pole 601, and a vehicle 602 appearin the reference image Ia illustrated in a section (a) of FIG. 10. Onthe V map VM, the road surface 600 in the reference image Ia correspondsto a road surface portion 600 a, the power pole 601 corresponds to apower pole portion 601 a, and the vehicle 602 corresponds to a vehicleportion 602 a.

Furthermore, the second generating unit 501 conducts linearapproximation on the position that is estimated to be a road surfacebased on the generated V map VM. When a road surface is flat,approximation is possible by using a single straight line; however, whenthe gradient of the road surface changes, there is a need to divide theV map VM into sections and conduct linear approximation with highaccuracy. Known technologies such as Hough transform or the least-squaremethod may be used as linear approximation. On the V map VM, the powerpole portion 601 a and the vehicle portion 602 a, which are clusterslocated above the detected road surface portion 600 a, are equivalent tothe power pole 601 and the vehicle 602, respectively, that are objectson the road surface 600. When a U-Disparity map is generated by thesecond generating unit 501 described later, only information above theroad surface is used to remove noise.

Furthermore, the second generating unit 501 generates a U map UM that isa U-Disparity map illustrated in a section (b) of FIG. 11 to recognizeobjects by using only information located above the road surfacedetected from the V map VM, i.e., by using information that is in adisparity image and that is equivalent to a left guardrail 611, a rightguardrail 612, a vehicle 613, and a vehicle 614 in the reference imageIa illustrated in a section (a) of FIG. 11. Here, the U map UM is atwo-dimensional histogram indicating the frequency distribution of thedisparity value dp, where the horizontal axis is the x axis of thereference image Ia and the vertical axis is the disparity value dp (ordistance) of the disparity image. The left guardrail 611 in thereference image Ia illustrated in FIG. the section (a) of 11 isequivalent to a left guardrail portion 611 a on the U map UM, the rightguardrail 612 is equivalent to a right guardrail portion 612 a, thevehicle 613 is equivalent to a vehicle portion 613 a, and the vehicle614 is equivalent to a vehicle portion 614 a.

Furthermore, the second generating unit 501 generates a U map UM_H thatis an example of the U-Disparity map illustrated in a section (c) ofFIG. 11 by using only information located above the road surfacedetected from the V map VM, i.e., by using information that is in adisparity image and that is equivalent to the left guardrail 611, theright guardrail 612, the vehicle 613, and the vehicle 614 in thereference image Ia illustrated in the section (a) of FIG. 11. Here, theU map UM_H, which is an example of the U-Disparity map, is an imagewhere the horizontal axis is the x axis of the reference image Ia, thevertical axis is the disparity value dp of the disparity image, and thepixel value is the height of an object. The left guardrail 611 in thereference image Ia illustrated in the section (a) of FIG. 11 isequivalent to a left guardrail portion 611 b on the U map UM_H, theright guardrail 612 is equivalent to a right guardrail portion 612 b,the vehicle 613 is equivalent to a vehicle portion 613 b, and thevehicle 614 is equivalent to a vehicle portion 614 b.

Furthermore, from the generated U map UM illustrated in a section (a) ofFIG. 12, the second generating unit 501 generates a real U map RM thatis a Real U-Disparity map illustrated in a section (b) of FIG. 12 inwhich the horizontal axis has been converted into the actual distance.Here, the real U map RM is a two-dimensional histogram in which thehorizontal axis is the actual distance in a direction from the imagingunit 10 b (the left camera) to the imaging unit 10 a (the right camera)and the vertical axis is the disparity value dp of the disparity image(or the distance in a depth direction that is converted from thedisparity value dp). The left guardrail portion 611 a on the U map UMillustrated in the section (a) of FIG. 12 is equivalent to a leftguardrail portion 611 c on the real U map RM, the right guardrailportion 612 a is equivalent to a right guardrail portion 612 c, thevehicle portion 613 a is equivalent to a vehicle portion 613 c, and thevehicle portion 614 a is equivalent to a vehicle portion 614 c.Specifically, on the U map UM, the second generating unit 501 does notdecimate pixels in the case of a long distance (the small disparityvalue dp) as an object is small and therefore there is a small amount ofdisparity information and a distance resolution is low but decimates alarge number of pixels in the case of a short distance as an objectappears to be large and therefore there is a large amount of disparityinformation and a distance resolution is high, thereby generating thereal U map RM that is equivalent to a plane view. As described later,the cluster of pixel values (object) (“isolated area” described later)is extracted from the real U map RM so that the object can be detected.In this case, the width of the rectangle enclosing a cluster correspondsto the width of an extracted object, and its height corresponds to thedepth of the extracted object. Furthermore, the second generating unit501 is capable of not only generating the real U map RM from the U mapUM but also generating the real U map RM directly from the disparityimage.

Furthermore, images input from the disparity-value deriving unit 3 tothe second generating unit 501 are not limited to the reference imageIa, but the comparison image Ib may be the target.

The second generating unit 501 is implemented by using the FPGA 51illustrated in FIG. 3. Furthermore, the second generating unit 501 maybe implemented when the CPU 52 executes programs stored in the ROM 53instead of the FPGA 51 that is a hardware circuit.

The clustering processing unit 502 is a functional unit that performsclustering processing to detect an object appearing in a disparity imageon the basis of each map output from the second generating unit 501. Asillustrated in FIG. 9, the clustering processing unit 502 includes anarea extracting unit 511 (extracting unit), a frame generating unit 512(determining unit), a first discarding unit 513, and an overlapprocessing unit 514.

The area extracting unit 511 is a functional unit that extracts anisolated area that is a cluster of pixel values from the real U map RMincluded in maps (images) output from the second generating unit 501.Specifically, the area extracting unit 511 conducts binarizationprocessing, labeling processing, or the like, on the real U map RM andextracts an isolated area with respect to each piece of identificationinformation on the labeling processing. For example, FIG. 13 illustratesa state where isolated areas are extracted from the real U map RM. Inthe example of the real U map RM illustrated in the case of FIG. 13, thearea extracting unit 511 extracts isolated areas 621 to 624 as isolatedareas. The isolated areas extracted by the area extracting unit 511correspond to objects appearing in the reference image Ia, and theyrepresent recognized areas of the objects in the reference image Ia.

Furthermore, based on the U map UM or the real U map RM generated by thesecond generating unit 501, the area extracting unit 511 is capable ofidentifying the position and the width (xmin, xmax) of the object at anisolated area in the x-axis direction on the disparity image and thereference image Ia. Furthermore, the area extracting unit 511 is capableof identifying the actual depth of an object based on information (dmin,dmax) on the height of the object on the U map UM or the real U map RM.Furthermore, based on the V map VM generated by the second generatingunit 501, the area extracting unit 511 is capable of identifying theposition and the height (ymin=“the y-coordinate that is equivalent tothe maximum height from the road surface with the maximum disparityvalue”, ymax=“the y-coordinate indicating the height of the road surfaceobtained from the maximum disparity value”) of an object in the y-axisdirection on the disparity image and the reference image Ia.Furthermore, the area extracting unit 511 is capable of identifying theactual size of an object in the x-axis direction and the y-axisdirection based on the width (xmin, xmax) of the object in the x-axisdirection, the height (ymin, ymax) in the y-axis direction, and thedisparity value dp that corresponds to each of them, identified on thedisparity image. As described above, by using the V map VM, the U mapUM, and the real U map RM, the area extracting unit 511 is capable ofidentifying the position and the actual width, height, and depth of theobject at an isolated area in the reference image Ia. Furthermore, asthe area extracting unit 511 identifies the position of an object in thereference image Ia, the position in a disparity image is determined, andthe distance to the object is also determined.

With regard to each extracted isolated area, the area extracting unit511 generates recognized-area information that is information about anisolated area and includes, in the recognized-area information, here forexample identification information on a labeling process and informationon the position and the size of an isolated area on the reference imageIa, the V map VM, the U map UM, and the real U map RM. The areaextracting unit 511 sends the generated recognized-area information tothe frame generating unit 512.

Furthermore, on an extracted isolated area, the area extracting unit 511may perform processing such as smoothing to reduce noise, disparitydispersion, and the like, which are present on the real U map RM, planedetection of the object at an isolated area, or deletion of unnecessaryareas.

The frame generating unit 512 is a functional unit that, with respect tothe isolated area of an object on the real U map RM extracted by thearea extracting unit 511, generates a frame at the object's area(hereafter, sometimes referred to as detection area) that is in adisparity image Ip (or the reference image Ia) and that corresponds tothe isolated area. Specifically, the frame generating unit 512 generatesdetection frames 631 a to 634 a in the disparity image Ip or thereference image Ia as illustrated in a section (b) of FIG. 14 such thatthey correspond to detection areas 631 to 634 that correspond to theisolated areas 621 to 624, respectively, which are extracted by the areaextracting unit 511 from the real U map RM, as illustrated in a section(a) of FIG. 14. The frame generating unit 512 includes the informationon the frame generated on the disparity image Ip or the reference imageIa in the recognized-area information and sends it to the firstdiscarding unit 513.

The first discarding unit 513 is a functional unit that determines whatthe object is on the basis of the actual size (width, height, depth) ofthe object (hereafter, sometimes referred to as detection object) in adetection area indicated with a frame by the frame generating unit 512based on the size of the detection area and that discards it inaccordance with the type of object. The first discarding unit 513 usesfor example the following (Table 1) to determine what a detection objectis. For example, when the width of the object is 1300 [mm], the heightis 1800 [mm], and the depth is 2000 [mm], it is determined that theobject is a “standard-sized automobile”. Here, the information thatrelates width, height, and depth with type of object (object type) maybe stored as a table like (Table 1) in the RAM 54, or the like. Here,the relation between a size and a type of object (object type)illustrated in (Table 1) is an example, and they may be defined as arelation between a different size and a type of object.

TABLE 1 Unit (mm) Object type Width Height Depth Motorbike, bicycle<1100 <2500 >1000 Pedestrian <1100 <2500 <=1000 Small-sized <1700 <1700<10000 automobile Standard-sized <1700 <2500 <10000 automobile Truck<3500 <3500 <15000 Others Not applied to above sizes

The first discarding unit 513 discards an object that is determined notto be targeted for subsequent processing (overlap processing, trackingprocessing, or the like, described later) in accordance with thedetermined type of detection object. For example, when pedestrians(persons) and vehicles are targeted for subsequent processing, the firstdiscarding unit 513 discards detection objects indicated by detectionframes 631 a, 632 a illustrated in the section (b) of FIG. 14 as theyare side wall objects (guardrails). To discard a detection object, forexample, the first discarding unit 513 includes a flag (discard flag)indicating discard in the recognized-area information on the detectionobject. Here, the first discarding unit 513 determines whether adetection object is to be discarded in accordance with the determinedtype of detection object; however, this is not a limitation, and it maybe determined whether an object in a detection area is to be discardedin accordance with the size of the detection area. The first discardingunit 513 includes a discard flag indicating whether the detection objectis to be discarded in the recognized-area information and sends it tothe overlap processing unit 514. Furthermore, with regard to a detectionobject in the following explanation of an overlap process and a trackingprocess, it is assumed that the discard flag included in therecognized-area information is off, that is, it is not discarded.

The overlap processing unit 514 is a functional unit that, whendetection areas are overlapped, performs an overlap process to determinewhether objects in the detection areas are to be discarded on the basisof the size of the overlapped detection areas. The overlap processingunit 514 includes a first determining unit 521, a distance calculatingunit 522 (first calculating unit), a second determining unit 523(determining unit), an overlapped-size calculating unit 524 (secondcalculating unit), a third determining unit 525, and a second discardingunit 526 (discarding unit).

The first determining unit 521 is a functional unit that determineswhether two detection areas are overlapped.

The distance calculating unit 522 is a functional unit that, when thefirst determining unit 521 determines that detection areas areoverlapped, calculates the distance (hereafter, sometimes referred to asthe distance between frames) between objects in the overlapped detectionareas in a depth direction.

The second determining unit 523 is a functional unit that determineswhether the distance between frames calculated by the distancecalculating unit 522 is less than a predetermined threshold. In thefollowing explanation, a distance equal to or longer than thepredetermined threshold is referred to as “long distance” (seconddistance range), and a distance less than the predetermined threshold is“short distance” (first distance range). Here, the second determiningunit 523 switches the predetermined threshold to be compared with thedistance between frames in accordance with the distance to a closerobject between two detection objects, for example, as illustrated in thefollowing (Table 2). For example, as illustrated in (Table 2), when thedistance to a closer object between two detection objects is equal toand more than 15 [m] and less than 35 [m], the second determining unit523 sets 4.5 [m] as the predetermined threshold to be compared with thedistance between frames. Here, the relation between the distance to adetection object and the threshold to be compared with the distancebetween frames illustrated in (Table 2) is an example, and they may bedefined with a different relation. The details of a determinationprocess by the second determining unit 523 are described later withreference to FIG. 19.

TABLE 2 Threshold item Threshold Distance between frames (distance to2.5 [m] detection object is less than 15[m]) Distance between frames(distance to 4.5 [m] detection object is equal to or more than 15[m] andless than 35[m]) Distance between frames (distance to   9 [m] detectionobject is more than 35[m])

Here, FIG. 15 illustrates an example of the case where the distancebetween frames is a short distance. A disparity image Ip1 illustrated inFIG. 15 indicates that a detection area 641 in which the detectionobject is a pedestrian and a detection area 642 in which the detectionobject is a vehicle is in a short distance and parts of the detectionareas 641, 642 are overlapped. Conversely, FIG. 16 illustrates anexample of the case where the distance between frames is a longdistance. A disparity image Ip2 illustrated in FIG. 16 indicates that adetection area 651 in which the detection object is a pedestrian and adetection area 652 in which the detection object is a vehicle are in along distance and parts of the detection areas 651, 652 are overlapped.

The overlapped-size calculating unit 524 is a functional unit thatcalculates the size (hereafter, sometimes referred to as overlap size)of the area where two detection areas are overlapped. The process tocalculate the overlap size by the overlapped-size calculating unit 524is explained later in detail with reference to FIGS. 19, 20, 22, and 23.

The third determining unit 525 is a functional unit that determineswhether the overlap size calculated by the overlapped-size calculatingunit 524 is more than a predetermined percentage of the size of any oneof the two detection areas (a threshold with regard to the overlappercentage of a detection area). Here, the third determining unit 525switches the predetermined percentage (threshold) depending on whetherthe distance between frames in two detection areas is a short distanceor a long distance, as illustrated in for example the following (Table3). For example, as illustrated in (Table 3), when the distance betweenframes in two detection areas is a long distance, the third determiningunit 525 uses 15[%] of the size of any one of the two detection areas asthe threshold with regard to the overlap percentage of the detectionareas. Here, the relation between the distance between frames and thethreshold with regard to the overlap percentage of detection areasillustrated in (Table 3) is an example, and they may be defined with adifferent relation. A determination process by the third determiningunit 525 is described later in detail with reference to FIG. 19.

TABLE 3 Threshold item Threshold Overlap percentage of detection areas35 [%] of any (when distance between frames is short) one Overlappercentage of detection areas 15 [%] of any (when distance betweenframes is long) one

The second discarding unit 526 is a functional unit that determineswhether objects in two detection areas are to be discarded in accordancewith a determination result regarding the overlap size by the thirddetermining unit 525. The second discarding unit 526 includes thediscard flag indicating whether the detection object is discarded in therecognized-area information and sends it to the tracking unit 503. Thediscard process by the second discarding unit 526 is described later indetail with reference to FIG. 19.

The area extracting unit 511, the frame generating unit 512, and thefirst discarding unit 513 of the clustering processing unit 502 and thefirst determining unit 521, the distance calculating unit 522, thesecond determining unit 523, the overlapped-size calculating unit 524,the third determining unit 525, and the second discarding unit 526 ofthe overlap processing unit 514, illustrated in FIG. 9, are implementedby using the FPGA 51 illustrated in FIG. 3. Furthermore, all or part ofthe area extracting unit 511, the frame generating unit 512, and thefirst discarding unit 513 of the clustering processing unit 502 and thefirst determining unit 521, the distance calculating unit 522, thesecond determining unit 523, the overlapped-size calculating unit 524,the third determining unit 525, and the second discarding unit 526 ofthe overlap processing unit 514 may be implemented when the CPU 52executes programs stored in the ROM 53 instead of the FPGA 51 that is ahardware circuit.

The tracking unit 503 is a functional unit that performs a trackingprocess on a detection object whose discard flag is off on the basis ofthe recognized-area information that is information related to theobject detected by the clustering processing unit 502. The tracking unit503 outputs the recognized-area information including a result of atracking process as recognition information to the vehicle controldevice 6 (see FIG. 3). The tracking unit 503 is implemented by using theFPGA 51 illustrated in FIG. 3. Furthermore, the tracking unit 503 may beimplemented when the CPU 52 executes programs stored in the ROM 53instead of the FPGA 51 that is a hardware circuit.

Furthermore, “the image processing apparatus” according to the presentinvention may be the clustering processing unit 502 or the recognitionprocessing unit 5 including the clustering processing unit 502.

Furthermore, the function of each functional unit of the recognitionprocessing unit 5 illustrated in FIG. 9 is illustrated as a concept, andthis configuration is not a limitation. For example, multiple functionalunits that are illustrated as separate functional units in therecognition processing unit 5 illustrated in FIG. 9 may be configured asa single functional unit. Conversely, a function provided in a singlefunctional unit in the recognition processing unit 5 illustrated in FIG.9 may be divided and configured as multiple functional units.

Operation of the Object Recognition Apparatus

Next, with reference to FIGS. 17 to 24, an explanation is given of aspecific operation of the object recognition apparatus 1.

Block Matching Processing of the Disparity-Value Deriving Unit

FIG. 17 is a flowchart that illustrates an example of operation duringblock matching processing by the disparity-value deriving unit accordingto the embodiment. With reference to FIG. 17, an explanation is given ofthe flow of operation during the block matching processing by thedisparity-value deriving unit 3 in the object recognition apparatus 1.

Step S1-1

The image acquiring unit 100 b in the disparity-value deriving unit 3captures an image of the object in the front by using the left camera(the imaging unit 10 b), generates analog image signals, and obtains aluminance image that is an image based on the image signals. Thus, imagesignals targeted for the subsequent image processing are obtained. Then,a transition is made to Step S2-1.

Step S1-2

The image acquiring unit 100 a in the disparity-value deriving unit 3captures an image of the object in the front by using the right camera(the imaging unit 10 a), generates analog image signals, and obtains aluminance image that is an image based on the image signals. Thus, imagesignals targeted for the subsequent image processing are obtained. Then,a transition is made to Step S2-2.

Step S2-1

The converting unit 200 b in the disparity-value deriving unit 3 removesnoise from the analog image signals obtained during capturing by theimaging unit 10 b and converts it into digital-format image data. Due tothis conversion into digital-format image data, image processing ispossible on the image based on the image data on a pixel by pixel basis.Then, a transition is made to Step S3-1.

Step S2-2

The converting unit 200 a in the disparity-value deriving unit 3 removesnoise from the analog image signals obtained during capturing by theimaging unit 10 a and converts it into digital-format image data. Due tothis conversion into digital-format image data, image processing ispossible on the image based on the image data on a pixel by pixel basis.Then, a transition is made to Step S3-2.

Step S3-1

The converting unit 200 b outputs the image based on the digital-formatimage data, converted at Step S2-1, as the comparison image Ib for blockmatching processing. Thus, the target image to be compared so as toobtain a disparity value during block matching processing is obtained.Then, a transition is made to Step S4.

Step S3-2

The converting unit 200 a outputs the image based on the digital-formatimage data, converted at Step S2-2, as the reference image Ia for blockmatching processing. Thus, the reference image to obtain a disparityvalue during block matching processing is obtained. Then, a transitionis made to Step S4.

Step S4

The cost calculating unit 301 of the disparity-value calculationprocessing unit 300 in the disparity-value deriving unit 3 calculatesand acquires the cost value C(p,d) of each of the candidate pixelsq(x+d,y) for the corresponding pixel on the basis of the luminance valueof the reference pixel p(x,y) in the reference image Ia and theluminance value of each of the candidate pixels q(x+d,y) that areidentified by shifting them from the pixel at the corresponding positionof the reference pixel p(x,y) by the shift amount d on the epipolar lineEL in the comparison image Ib based on the reference pixel p(x,y).Specifically, during block matching processing, the cost calculatingunit 301 calculates, as the cost value C, the degree of dissimilaritybetween the reference area pb that is a predetermined area with thereference pixel p in the reference image Ia as a center and thecandidate area qb (the same size as the reference area pb) with thecandidate pixel q in the comparison image Ib as a center. Then, atransition is made to Step S5.

Step S5

The determining unit 302 of the disparity-value calculation processingunit 300 in the disparity-value deriving unit 3 determines that theshift amount d that corresponds to the minimum value of the cost value Ccalculated by the cost calculating unit 301 is the disparity value dpwith respect to a pixel in the reference image Ia targeted forcalculation of the cost value C. Then, the first generating unit 303 ofthe disparity-value calculation processing unit 300 in thedisparity-value deriving unit 3 generates a disparity image that is animage representing the luminance value of each pixel of the referenceimage Ia with the disparity value dp that corresponds to the pixel onthe basis of the disparity value dp determined by the determining unit302. The first generating unit 303 outputs the generated disparity imageto the recognition processing unit 5.

Although block matching processing is explained above as an example ofstereo matching processing, this is not a limitation, and SGM(Semi-Global Matching) technique may be used for processing.

Object Recognition Process of the Recognition Processing Unit

FIG. 18 is a flowchart that illustrates an example of operation duringthe object recognition process by the recognition processing unitaccording to the embodiment. FIG. 19 is a flowchart that illustrates anexample of operation during the overlap process by the recognitionprocessing unit according to the embodiment. FIG. 20 is a diagram thatillustrates an overlap size when the distance between frames is a shortdistance. FIG. 21 is a diagram that illustrates operation to discard adetection object when the distance between frames is a short distance.FIG. 22 is a diagram that illustrates an overlap size when the distancebetween frames is a long distance. FIG. 23 is a diagram that illustratesa case where there is no overlap size when the distance between framesis a long distance. FIG. 24 is a diagram that illustrates a case where adetection object is not discarded when the distance between frames is along distance. With reference to FIGS. 18 to 24, an explanation is givenof the flow of operation during the object recognition process by therecognition processing unit 5 in the object recognition apparatus 1.

Step S11

The second generating unit 501 receives the disparity image Ip from thedisparity-value calculation processing unit 300, receives the referenceimage Ia from the disparity-value deriving unit 3, and generates variousimages, such as the V map VM, the U map UM, the U map UM_H, and the realU map RM. Then, a transition is made to Step S12.

Step S12

The area extracting unit 511 of the clustering processing unit 502extracts an isolated area that is a cluster of pixel values from thereal U map RM included in the maps (images) output from the secondgenerating unit 501. Furthermore, by using the V map VM, the U map UM,and the real U map RM, the area extracting unit 511 identifies theposition of the object at an isolated area and the actual width, height,and depth in the reference image Ia or the disparity image Ip. Then, foreach extracted isolated area, the area extracting unit 511 generatesrecognized-area information that is information about an isolated areaand here includes, in the recognized-area information, for example theidentification information on labeling processing and information suchas the position and the size of an isolated area in the reference imageIa, the V map VM, the U map UM, and the real U map RM. The areaextracting unit 511 sends the generated recognized-area information tothe frame generating unit 512. Then, a transition is made to Step S13.

Step S13

The frame generating unit 512 of the clustering processing unit 502 is afunctional unit that, with regard to the isolated area of an object onthe real U map RM extracted by the area extracting unit 511, generates aframe for the detection area of the object that corresponds to theisolated area in the disparity image Ip (or the reference image Ia). Theframe generating unit 512 includes the information on the framegenerated on the disparity image Ip or the reference image Ia in therecognized-area information and sends it to the first discarding unit513. Then, a transition is made to Step S14.

Step S14

The first discarding unit 513 of the clustering processing unit 502determines what the object is on the basis of the actual size (width,height, depth) of the detection object in a detection area based on thesize of the detection area indicated with the frame by the framegenerating unit 512 and discards it in accordance with the type ofobject. To discard a detection object, for example, the first discardingunit 513 includes a flag (discard flag) indicating discard in therecognized-area information on the detection object. The firstdiscarding unit 513 includes the discard flag indicating whether thedetection object is to be discarded in the recognized-area informationand sends it to the overlap processing unit 514. Then, a transition ismade to Step S15.

Step S15

When detection areas are overlapped, the overlap processing unit 514performs an overlap process to determine whether objects in thedetection areas are to be discarded on the basis of the size of theoverlapped detection areas. The overlap process by the overlapprocessing unit 514 is explained with reference to FIG. 19.

Step S151

The first determining unit 521 of the overlap processing unit 514identifies any two detection objects among the detection objects thatcorrespond to pieces of recognized-area information received from thefirst discarding unit 513. Then, a transition is made to Step S152.

Step S152

The first determining unit 521 determines whether the detection areas ofthe two identified detection objects are overlapped. When the twodetection areas are overlapped (Step S152: Yes), a transition is made toStep S153, and when they are not overlapped (Step S152: No), Step S151is returned so that the first determining unit 521 identifies twodifferent detection objects.

Step S153

When the first determining unit 521 determines that the detection areasare overlapped, the distance calculating unit 522 of the overlapprocessing unit 514 calculates the distance between frames of theobjects in the overlapped detection areas in a depth direction. Then, atransition is made to Step S154.

Step S154

The second determining unit 523 of the overlap processing unit 514determines whether the distance between frames calculated by thedistance calculating unit 522 is less than a predetermined threshold.When the distance between frames is less than the predeterminedthreshold, that is, when the distance between frames is a short distance(Step S154: Yes), a transition is made to Step S155, and when it isequal to or more than the predetermined threshold, that is, when thedistance between frames is a long distance (Step S154: No), a transitionis made to Step S159.

Step S155

When the second determining unit 523 determines that the distancebetween frames is a short distance, the overlapped-size calculating unit524 of the overlap processing unit 514 calculates the overlap size ofthe area where two detection areas are overlapped. For example, asillustrated in FIG. 20, when a detection area 661 and a detection area662 are overlapped, the overlapped-size calculating unit 524 calculatesthe size of an overlapped area 663 that is an area overlapped by using(height OL_H)×(width OL_W). Then, a transition is made to Step S156.

Step S156

The third determining unit 525 of the overlap processing unit 514determines whether the overlap size calculated by the overlapped-sizecalculating unit 524 is equal to or more than a predetermined percentageof the size of any one of the two detection areas (a threshold withregard to the overlap percentage of the detection areas). When theoverlap size is equal to or more than the predetermined percentage ofthe size of any one of the two detection areas (Step S156: Yes), atransition is made to Step S157, and when it is less than thepredetermined percentage (Step S156: No), a transition is made to StepS158.

Step S157

When both the detection objects are vehicles, the second discarding unit526 of the overlap processing unit 514 does not discard the detectionobject in a short distance with a high degree of importance as thetarget for a tracking process but discards the detection object in along distance. The second discarding unit 526 includes the discard flagindicating non-discard in the recognized-area information on thedetection object in a short distance, includes the discard flagindicating discard in the recognized-area information on the detectionobject in a long distance, and sends them to the tracking unit 503.

Conversely, when one of the two detection objects is a vehicle and theother one is not a vehicle and is an object whose size is smaller than avehicle, the second discarding unit 526 does not discard the detectionobject that is a vehicle but discards the detection object that is not avehicle and has a size smaller than vehicles. There is a highpossibility that a detection object that is not a vehicle and has a sizesmaller than a vehicle is, for example, part of the vehicle that isimproperly detected as a pedestrian and therefore it is discarded. Forexample, as illustrated in FIG. 21, when the distance between frames inthe detection area indicated by a detection frame 671 and the detectionarea indicated by a detection frame 672 is a short distance, and whenthe detection object indicated by the detection frame 671 is a vehicleand the detection object indicated by the detection frame 672 is anobject other than vehicles (a person in the vehicle in FIG. 21), thesecond discarding unit 526 does not discard the vehicle indicated by thedetection frame 671 but discards the detection object indicated by thedetection frame 672. The second discarding unit 526 includes the discardflag indicating non-discard in the recognized-area information on thedetection object that is a vehicle, includes the discard flag indicatingdiscard in the recognized-area information on the detection object thatis not a vehicle, and sends it to the tracking unit 503.

Step S158

When the third determining unit 525 determines that the overlap size issmaller than the predetermined percentage of the size of any one of thetwo detection areas, the second discarding unit 526 determines that theobjects in the detection areas have a high degree of importance as thetarget for a tracking process and does not discard any of the detectionobjects. The second discarding unit 526 includes the discard flagindicating non-discard in the recognized-area information on each of thetwo detection objects and sends it to the tracking unit 503.

Step S159

When the second determining unit 523 determines that the distancebetween frames is a long distance, the overlapped-size calculating unit524 calculates a central area (an example of a partial area) of thedetection area with the detection object in a short distance, includedin the two detection areas. Specifically, as illustrated in FIG. 22, theoverlapped-size calculating unit 524 calculates for example a centralarea 681 a that has the size of a central area in a horizontal direction(e.g., an area with 80[%] of the width in a horizontal direction) withregard to the detection area 681 whose detection object is closer,included in the two detection areas 681, 682. Although theoverlapped-size calculating unit 524 calculates the central area of thedetection area with the detection object in a short distance, this isnot a limitation and, for example, the area with a predeterminedpercentage (e.g., 85[%]) from the extreme right of the detection areamay be calculated. Then, a transition is made to Step S160.

Step S160

The overlapped-size calculating unit 524 calculates the overlap size ofthe area where the central area of the detection area with the detectionobject in a short distance and the detection area with the detectionobject in a long distance are overlapped, included in the two detectionareas. For example, as illustrated in FIG. 22, when the central area 681a of the detection area 681 is overlapped with the detection area 682,the overlapped-size calculating unit 524 calculates the size of anoverlapped area 683 that is an area overlapped by using (heightOL_H1)×(width OL_W1). Then, a transition is made to Step S161.

Step S161

The third determining unit 525 determines whether the overlap sizecalculated by the overlapped-size calculating unit 524 is equal to ormore than a predetermined percentage (a threshold with regard to anoverlap percentage) of the size of any one of the central area of thedetection area with the detection object in a short distance and thedetection area with the detection object in a long distance. When theoverlap size is equal to or more than the predetermined percentage ofthe size of any one of them (Step S161: Yes), a transition is made toStep S162, and when it is less than the predetermined percentage (StepS161: No), a transition is made to Step S163.

Step S162

With respect to two detection objects, the second discarding unit 526does not discard the detection object in a short distance with a highdegree of importance as the target for a tracking process but discardsthe detection object in a long distance. In the example illustrated inFIG. 22, when the size (overlap size) of the overlapped area 683 isequal to or more than the predetermined percentage of the size of thecentral area 681 a or the detection area 682, the second discarding unit526 does not discard the detection object in the detection area 681 thatis in a short distance but discards the detection object in thedetection area 682 that is in a long distance. The second discardingunit 526 includes the discard flag indicating non-discard in therecognized-area information on the detection object in a short distance,includes the discard flag indicating discard in the recognized-areainformation on the detection object in a long distance, and sends themto the tracking unit 503.

Step S163

When the third determining unit 525 determines that the overlap size isless than the predetermined percentage of the size of any one of thecentral area of the detection area with the detection object in a shortdistance and the detection area with the detection object in a longdistance, the second discarding unit 526 determines that the objects inboth the detection areas have a high degree of importance as the targetfor a tracking process and does not discard any of the detectionobjects. That is, when it is simply determined that the overlap size oftwo detection areas is equal to or more than the predeterminedpercentage of the size of any one of the two detection areas, there is apossibility that the detection object in a long distance is discarded;however, as the overlap size is obtained with respect to the centralarea of the detection area in a short distance, it is possible toprevent a detection object (e.g., pedestrian) in a long distance whichshould not be discarded from being discarded although the detectionareas are overlapped near the end. The second discarding unit 526includes the discard flag indicating non-discard in the recognized-areainformation on each of the two detection objects and sends it to thetracking unit 503.

For example, in the example illustrated in FIG. 23, although the twodetection areas 681, 682 a are overlapped, the central area 681 a of thedetection area 681 is not overlapped with the detection area 682 a, andtherefore the third determining unit 525 determines that the overlapsize is less than the predetermined percentage of the size of any one ofthe central area of the detection area of the detection object in ashort distance and the detection area of the detection object in a longdistance. In this case, the second discarding unit 526 determines thatthe detection objects in both the detection areas 681, 682 a have a highdegree of importance as the target for a tracking process and does notdiscard any of the detection objects.

Furthermore, as illustrated in FIG. 24, for example, when the distancebetween frames in the detection area indicated by a detection frame 691and the detection area indicated by the detection frame 692 is a longdistance and the third determining unit 525 determines that the overlapsize is less than the predetermined percentage of the size of any one ofthe central area of the detection area in the detection frame 691 withthe detection object in a short distance and the detection area in thedetection frame 692 with the detection object in a long distance, thesecond discarding unit 526 does not discard the detection objectsindicated by the detection frames 691, 692.

After the process at Step S157, S158, S162, or S163 is finished, atransition is made to Step S16.

Step S16

The tracking unit 503 performs a tracking process on a detection objectwhose discard flag is off on the basis of the recognized-areainformation that is information about an object detected by theclustering processing unit 502. The tracking unit 503 outputs therecognized-area information including a result of the tracking processas recognition information to the vehicle control device 6 (see FIG. 3).

As described above, the object recognition process is conducted duringthe process at Steps S11 to S16 illustrated in FIG. 18, and, at StepS15, the overlap process is conducted during the process at Steps S151to S163 illustrated in FIG. 19.

As described above, the distance between frames of the detection areasof two detected objects is calculated, the method of calculating thesize of the overlapped area with respect to the detection areas of thetwo objects is switched in accordance with the distance between frames,and it is determined whether the detection object is to be discarded inaccordance with the size. Thus, a discard process may be properlyconducted. That is, according to the present embodiment, it is possibleto discard objects that need to be discarded and refrain from discardingobjects that do not need to be discarded other than vehicles.

Furthermore, when the distance between frames is a long distance, thecentral area of the detection area with the detection object in a shortdistance, included in the two detection areas, is calculated, theoverlap size of the area where the central area is overlapped with thedetection area with the detection object in a long distance iscalculated, it is determined whether it is equal to or more than thepredetermined percentage of the size of any one of the central area andthe detection area with the detection object in a long distance, andwhen it is less than that, the two detection objects are not discarded.Thus, when it is simply determined whether the overlap size of twodetection areas is equal to or more than the predetermined percentage ofthe size of any one of the two detection areas, there is a possibilitythat the detection object in a long distance is discarded; however, asthe overlap size is obtained with respect to the central area of thedetection area in a short distance, it is possible to prevent adetection object (e.g., pedestrian) in a long distance which should notbe discarded from being discarded although the detection areas areoverlapped near the end.

Furthermore, when the distance between frames is a short distance, thesize of the area where the two detection areas are overlapped iscalculated, it is determined whether it is equal to or more than thepredetermined percentage of the size of any one of the two detectionareas, and when it is equal to or more than that and when one of the twodetection objects is a vehicle and the other one is not a vehicle and itis an object smaller than a vehicle, the detection object that is avehicle is not discarded and the detection object that is not a vehicleand is smaller than a vehicle is discarded. Thus, objects that are notvehicles may be discarded accurately as there is a high possibility offalse detection.

Furthermore, according to the above-described embodiment, the cost valueC is an evaluation value representing a degree of dissimilarity;however, it may be an evaluation value representing a degree ofsimilarity. In this case, the shift amount d with which the cost valueC, the degree of similarity, becomes maximum (extreme value) is thedisparity value dp.

Furthermore, according to the above-described embodiment, although theobject recognition apparatus 1 installed in an automobile that is thevehicle 70 is explained, this is not a limitation. For example, it maybe installed in other examples of vehicles, such as bikes, bicycles,wheelchairs, or cultivators for agricultural use. Furthermore, it may benot only a vehicle that is an example of a movable body, but also amovable body such as a robot.

Furthermore, according to the above-described embodiment, when at leastany of functional units of the disparity-value deriving unit 3 and therecognition processing unit 5 in the object recognition apparatus 1 isimplemented by executing a program, the program is provided by beingpreviously installed in a ROM, or the like. Furthermore, a configurationmay be such that a program executed by the object recognition apparatus1 according to the above-described embodiment is provided by beingstored, in the form of a file that is installable and executable, in arecording medium readable by a computer, such as a CD-ROM, a flexibledisk (FD), a CD-R (compact disk recordable), or a DVD (digital versatiledisk). Furthermore, a configuration may be such that the programexecuted by the object recognition apparatus 1 according to theabove-described embodiment is stored in a computer connected via anetwork such as the Internet and provided by being downloaded via thenetwork. Moreover, a configuration may be such that the program executedby the object recognition apparatus 1 according to the above-describedembodiment is provided or distributed via a network such as theInternet. Furthermore, the program executed by the object recognitionapparatus 1 according to the above-described embodiment has a modularconfiguration that includes at least any of the above-describedfunctional units, and in terms of actual hardware, the CPU 52 (the CPU32) reads the program from the above-described ROM 53 (the ROM 33) andexecutes it so as to load and generate the above-described functionalunits in a main storage device (the RAM 54 (the RAM 34), or the like).

The above-described embodiments are illustrative and do not limit thepresent invention. Thus, numerous additional modifications andvariations are possible in light of the above teachings. For example, atleast one element of different illustrative and exemplary embodimentsherein may be combined with each other or substituted for each otherwithin the scope of this disclosure and appended claims. Further,features of components of the embodiments, such as the number, theposition, and the shape are not limited the embodiments and thus may bepreferably set. It is therefore to be understood that within the scopeof the appended claims, the disclosure of the present invention may bepracticed otherwise than as specifically described herein.

The method steps, processes, or operations described herein are not tobe construed as necessarily requiring their performance in theparticular order discussed or illustrated, unless specificallyidentified as an order of performance or clearly identified through thecontext. It is also to be understood that additional or alternativesteps may be employed.

Further, any of the above-described apparatus, devices or units can beimplemented as a hardware apparatus, such as a special-purpose circuitor device, or as a hardware/software combination, such as a processorexecuting a software program.

Further, as described above, any one of the above-described and othermethods of the present invention may be embodied in the form of acomputer program stored in any kind of storage medium. Examples ofstorage mediums include, but are not limited to, flexible disk, harddisk, optical discs, magneto-optical discs, magnetic tapes, nonvolatilememory, semiconductor memory, read-only-memory (ROM), etc.

Alternatively, any one of the above-described and other methods of thepresent invention may be implemented by an application specificintegrated circuit (ASIC), a digital signal processor (DSP) or a fieldprogrammable gate array (FPGA), prepared by interconnecting anappropriate network of conventional component circuits or by acombination thereof with one or more conventional general purposemicroprocessors or signal processors programmed accordingly.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. Processing circuitryincludes a programmed processor, as a processor includes circuitry. Aprocessing circuit also includes devices such as an application specificintegrated circuit (ASIC), digital signal processor (DSP), fieldprogrammable gate array (FPGA) and conventional circuit componentsarranged to perform the recited functions.

What is claimed is:
 1. An information processing apparatus, comprising:calculating circuitry configured to calculate a distance between twoobject, detected based on distance information on the objects that areoverlapped in detection areas of the objects, in a depth direction inthe detection areas; and discarding circuitry configured to determinewhether at least one of the two objects in the detection areas is to bediscarded by using a method that corresponds to the distance calculatedby the calculating circuitry, wherein the calculating circuitryincludes: a first calculating circuitry configured to calculate adistance between two objects, detected based on distance information onthe objects, in a depth direction in detection areas of the objects; anda second calculating circuitry configured to calculate an overlap sizethat is a size of an overlapped area with regard to the two detectionareas by using a method that corresponds to the distance calculated bythe first calculating circuitry, and wherein the discarding circuitry isconfigured to determine whether the at least one of the two objects inthe detection areas is to be discarded in accordance with the overlapsize calculated by using the method that corresponds to the distance. 2.The information processing apparatus according to claim 1 furthercomprising: determining circuitry configured to determine whether thedistance calculated by the first calculating circuitry is included in afirst distance range or included in a second distance range that isfarther than the first distance range, wherein when the determiningcircuitry determines that the distance is included in the seconddistance range, the second calculating circuitry calculates, as theoverlap size, a size of an area where a partial area of the detectionarea of a closer one of the objects is overlapped with the detectionarea of a farther one of the objects, included in the two detectionareas, and when the overlap size is less than a predetermined percentageof a size of any one of the partial area and the detection area of thefarther one of the objects, the discarding circuitry discards neitherthe closer one of the objects nor the farther one of the objects.
 3. Theinformation processing apparatus according to claim 2, wherein when thedetermining circuitry determines that the distance is included in thesecond distance range, the second calculating circuitry obtains, as thepartial area, a predetermined central area in a horizontal direction ofthe detection area of the closer one of the objects and calculates, asthe overlap size, a size of an area where the central area is overlappedwith the detection area of the farther one of the objects.
 4. Theinformation processing apparatus according to claim 2, wherein when theoverlap size is equal to or more than the predetermined percentage ofthe size of any one of the partial area and the detection area of thefarther one of the objects, the discarding circuitry does not discardthe closer one of the objects but discards the farther one of theobjects.
 5. The information processing apparatus according to claim 2,wherein the determining circuitry determines that the distance isincluded in the first distance range, the second calculating circuitrycalculates, as the overlap size, a size of an area where the twodetection areas are overlapped, and when the overlap size is equal to ormore than a predetermined percentage of a size of any one of the twodetection areas and when one of the two detection areas represents avehicle and another one represents an object other than a vehicle, thediscarding circuitry does not discard an object that is a vehicle butdiscards an object that is other than a vehicle.
 6. The informationprocessing apparatus according to claim 5, wherein when the overlap sizeis equal to or more than a predetermined percentage of a size of any oneof the two detection areas and when both the detection areas represent avehicle, the discarding circuitry does not discard the closer one of theobjects, included in the objects indicated by the two detection areas,but discards the farther one of the objects.
 7. The informationprocessing apparatus according to claim 5, wherein when the overlap sizeis less than a predetermined percentage of a size of any one of the twodetection areas, the discarding circuitry discards neither the closerone of the objects nor the farther one of the objects, included in theobjects indicated by the two detection areas.
 8. The informationprocessing apparatus according to claim 1, further comprising:extracting circuitry configured to extract an isolated area indicatingan object based on the distance information; and determining circuitryconfigured to determine the detection area by generating a frame for theisolated area.
 9. An object recognition apparatus comprising: firstimaging circuitry configured to obtain a first captured image bycapturing an image of an object; second imaging circuitry that islocated at a position different from a position of the first imagingcircuitry and that is configured to obtain a second captured image bycapturing an image of the object; generating circuitry configured togenerate the distance information in accordance with a disparity valueobtained from the first captured image and the second captured imagewith respect to the object; and the information processing apparatusaccording to claim
 1. 10. A device control system comprising: the objectrecognition apparatus according to claim 9; and control circuitryconfigured to control a control target based on information about anobject detected by the object recognition apparatus.
 11. A movable bodycomprising the device control system according to claim
 10. 12. Theinformation processing apparatus according to claim 1, wherein: thediscarding circuitry configured to determine performs a determining ofwhether each of the two objects in the detection areas is to bediscarded.
 13. An image processing method comprising: calculating adistance between two objects, detected based on distance information onthe objects that are overlapped in detection areas of the objects, in adepth direction in the detection areas; and discarding by determiningwhether at least one of the objects in the detection areas is to bediscarded by using a method that corresponds to the distance calculatedand discarding the at least one of the objects which is determined to bein the detection area, wherein the calculating includes: a firstcalculating of a distance between two objects, detected based ondistance information on the objects, in a depth direction in detectionareas of the objects; and a second calculating of an overlap size thatis a size of an overlapped area with regard to the two detection areasby using a method that corresponds to the distance calculated by thefirst calculating, and wherein the discarding is configured to determinewhether the at least one of the objects in the detection areas is to bediscarded in accordance with the overlap size that has been calculated.14. The method according to claim 13, further comprising: determiningwhether the distance calculated by the first calculating is included ina first distance range or included in a second distance range that isfarther than the first distance range, wherein when the determiningwhether the distance calculated determines that the distance is includedin the second distance range, the second calculating calculates, as theoverlap size, a size of an area where a partial area of the detectionarea of a closer one of the objects is overlapped with the detectionarea of a farther one of the objects, included in the two detectionareas, and when the overlap size is less than a predetermined percentageof a size of any one of the partial area and the detection area of thefarther one of the objects, the discarding discards neither the closerone of the objects nor the farther one of the objects.
 15. The methodaccording to claim 14, wherein when the determining determines that thedistance is included in the second distance range, the secondcalculating obtains, as the partial area, a predetermined central areain a horizontal direction of the detection area of the closer one of theobjects and calculates, as the overlap size, a size of an area where thecentral area is overlapped with the detection area of the farther one ofthe objects.
 16. The method according to claim 14, wherein when theoverlap size is equal to or more than the predetermined percentage ofthe size of any one of the partial area and the detection area of thefarther one of the objects, the discarding does not discard the closerone of the objects but discards the farther one of the objects.
 17. Themethod according to claim 13, wherein: the discarding performs adetermining of whether each of the two objects in the detection areas isto be discarded.
 18. A non-transitory computer-readable recording mediumthat contains a computer program that causes a computer to execute:calculating a distance between two objects, detected based on distanceinformation on the objects that are overlapped in detection areas of theobjects, in a depth direction in the detection areas; and discarding bydetermining whether at least one of the objects in the detection areasis to be discarded by using a method that corresponds to the distancecalculated and discarding the at least one of the objects which isdetermined to be in the detection area, wherein the calculatingincludes: a first calculating of a distance between two objects,detected based on distance information on the objects, in a depthdirection in detection areas of the objects; and a second calculating ofan overlap size that is a size of an overlapped area with regard to thetwo detection areas by using a method that corresponds to the distancecalculated by the first calculating, and wherein the discarding includesdetermining whether the at least one of the objects in the detectionareas is to be discarded in accordance with the overlap size that hasbeen calculated.
 19. The non-transitory computer-readable recordingmedium according to claim 18, wherein: the discarding performs adetermining of whether each of the two objects in the detection areas isto be discarded.